Background Jobs view

Administration System Background Jobs Background Jobs view

Prerequisites

• Background Jobs - View privilege.

The Background Jobs view displays all the background jobs running in the cluster, including the background jobs that you launch from the Actions menu in this view and any reporting, EDI, or Groovy jobs. In addition to these jobs, each yard has a heartbeat background job that runs automatically in the background. You do not need to launch this job. The Background Jobs view also displays other background jobs that run occasionally, such as sending a notification request. N4 displays these jobs in the list view only while they are running. When completed, it removes the jobs from the list view. 

Background Jobs that appear with a bullet under Execute in Cluster are saved to the database. By default, Execute in Cluster is the last column of the view.

Before you begin:

You must set up job groups that are for the specific nodes you want the background jobs to run on. See Job Groups view (on page 1) for more information. The background jobs that are accessed through the Background Jobs view Actions menu, as well as the EDI and Groovy background jobs, can only run on a single node, therefore, the Job Groups you create for these background jobs can only include one node. Background jobs for N4 Automation are not restricted to running on one node (see below).

It is possible to run multiple background jobs on a single node, but it's always a best practice to test this prior to production.

When a node in a job group becomes unresponsive for over three minutes, it fails over (or switches) to run all the jobs on a different node.

To start background jobs:

• Select Actions, then the background job you want to start. You will be asked to select the Job Group for the node the background job should run on. After you use the Actions menu to launch a job, you must click to view the job in the list view.

• If you're restarting a background job after manually halting the job, or if the processing node has gone down, any incomplete or repeating jobs that are saved in the database (as indicated by the bullet in the Execute in Cluster column) are restarted on that node, automatically. If a job is scheduled as On Demand, N4 does not launch it automatically after a system restart.

• For EDI and Groovy jobs, see EDI Jobs view (on page 1) and Groovy Jobs view for instructions on how to schedule or manually start these jobs.

To stop a background job:

• Select the background job running, then select Actions Halt Job. 

To diagnose a background job problem from the Background Jobs view:

From the Background Jobs (on page 1) view, right-click on the header bar and add the Status Details column header.

The Status Details column contains a long string of logged details regarding each background job. You can use this information for troubleshooting.

Status Details are only displayed for background jobs saved to the database, indicated by a bullet in the Execute in Cluster column.

To recover if a node goes down:

To recover, stop the background job by selecting Actions Halt Job. Restart the job using a different Job Group that's associated with a different node. If a node goes down, N4 changes the Job Group Status of jobs running on that node as follows:

• All Nodes are down: All the nodes in the job group are down.

• Some nodes are down: Some but not all of the nodes in the job group are down.

To recover when the Job Status is "Job has multiple triggers"

A network outage or a slow node can cause a job to be restarted, even if the job is still running. The job will have the status "Job has multiple triggers" indicating an error where two instances of the job may be running which can cause other problems. When this happens, halt the job (Actions Halt Job), then reschedule it. For example, if this was an EDI job, go to EDI Jobs to restart the job after halting it.

If this condition recurs it may indicate a bigger problem with the system. Contact Navis support and open a case. Attach the following information prior to halting and restarting the job, as long as operations will allow it:

• A dump of all the qrtz_ tables in the database

• Screenshot of the Background Jobs view with the Status

• GC logs from all N4 nodes

• The navis-apex.log files from all N4 nodes for the few hours leading up to the error

Automatic recovery if a node goes down:

It is possible that a background job can automatically recover if it runs into issues, such as when the heartbeat from a node is not getting through or you kill the node when shutdown takes too long.

A background job that automatically recovers will have a _recovered suffix added to its name, such as htsched01_recovered. If you see this you can leave the job running per normal and you only need to perform a refresh when a problem is identified, such as a job is hanging or no scheduling is occurring.

Background Jobs for N4 Automation

Background Jobs for N4 Automation are not restricted to running on one node. For instructions for how to set up job groups for the various N4 Automation schedulers, see Set up load balancing (job groups) for schedulers in the Navis N4 Automation Administrator and User Guide.

The AGV, ASC and CHE dispatcher jobs also contain a flag for the Quartz scheduler to delete the job and the trigger, then reschedule a new job based on the original job. This flag can be disabled by applying this Java option to each Tomcat application node:

-Dcom.navis.framework.quartz.rescheduleIfRecoveryEnabled=false

Background Jobs Columns

For XPS operations

N4 displays the xpsCacheMasterHeartbeat.XPS:<scope><(yard gkey)>.<timestamp> jobs in the list view. It displays one job for each yard that had the XPS State as READY, when the N4 server started. The timestamp displayed in the job name indicates that the connection between the N4 server and XPS server is live.

You can monitor the connection between the N4 servers and the XPS server using the Cluster Services view (on page 1).

For EDI operations

The list view displays the scheduled EDI jobs as <scope>/<schedule name> at all times, until the EDI Job is unscheduled or halted. For more information on EDI jobs, see EDI Jobs view (on page 1).

Scope level for background jobs

These are guidelines for the scope level you should use for running the various background jobs.

Background Jobs view - Actions menu

The Actions menu lists the various background jobs that you can manually start on any node in the cluster. When you select a background job from the Actions menu, N4 displays the Node Selection form (on page 1). Use the Node Selection form to start a background job on a specific node in the cluster.

Each action in the Background Jobs view is secured by a corresponding privilege. You must assign the appropriate privilege to user roles (on page 1) to enable the users to run a background job.

You can use the Actions menu to launch the following background jobs:

• Launch Unit Maintenance Jobs (on page 1)

• Launch Road Call-Up (on page 1)

• Launch Database Purge/Archive (on page 1)

• Launch Event Billing Extract (on page 1)

• Launch Marine Event Billing Extract (on page 1)

• Launch Appointment Event Billing Extract (on page 1)

• Launch Vessel Statistics Job

• Launch CHE Statistics Job

• Launch Secondary Statistics Job

• Launch Xps Data Harvester Job

• Launch Auto-Create Dashboard Job

• Launch Bento Server (ECN4 support) (on page 1)

• Launch Yard Block Statistics Job

• Launch Auto Close Truck Visit Job (on page 1)

• Launch Truck Visit Timing Check Job (on page 1)

• Launch Auto Expire Appointment Job (on page 1)

• Launch Auto Late Appointment Job (on page 1)

• Launch Auto Canceled not reused Appointment Job (on page 1)

• Launch Optimal Transfer Zone Slot Finder Job (on page 1)

• Launch Update Yard WI Estimated Move Time Job (on page 1)

• Launch Update Rail WI Estimated Move Time Job (on page 1)

• Launch Auto Close Departed Vessel Visits Job (on page 1)

• Launch LDC Kafka Lag Monitor Job (on page 1)

• Halt Job (on page 1)